The present invention relates to a digital image processing technique. More particularly, the present invention relates to a technique of inserting identification data (electronic watermark data) having special information into digital images.
Recently, illegal copies of digital images have raised troublesome questions. In order to prevent such illegal replication, the system has been considered that encrypts digital image data and allows only the reproduction system with a valid secret decryption key to reproduce encrypted digital image data.
However, after the encryption is once decoded, this system cannot prevent subsequent replication. In order to prevent digital images from being illegally used or replicated, the method has been considered of directly burying special information (hereinafter referred to as “electronic watermark data”) in digital images.
Two types of data including visible electronic watermark data and invisible electronic watermark data are considered as electronic watermark data for digital images. The visible electronic watermark data, which contains special characters or symbols combined with an image, can be visually sensed. This electronic watermark data may degrade the image quality but has the advantage of visually warning users to prevent misappropriation of digital images.
An example of burying such visible electronic watermark data is disclosed in JP-A-241403/1996. In this method, only the luminance of pixels corresponding to opaque portions of electronic watermark data is varied so that visible electronic watermark data is synthesized with the original image without changing the color components. The scaling value of varying the pixel luminance component depends on color components, random numbers, pixel values of electronic watermark data, or others.
The invisible electronic watermark data is buried in an image, in consideration of degradation of image quality. Since the image quality degradation is not substantially negligible, the invisible watermark data cannot be visually recognized. However, if special information, which can be recognized by an author, is buried as the electronic watermark data, the author can specify by detecting the electronic watermark data even after illegal replication.
Moreover, information about replication disapproval may be buried in an image. In such a case, when the reproduction unit, for example, detects the replication disapproval information, the reproduction by a VTR or the equivalent can be restricted by informing the user that the detected information is reproduction prohibited data, or by operating the replication preventing mechanism within the reproduction unit.
Prior various methods have been proposed to bury invisible electronic watermark data into digital images.
For example, one approach is to bury special information as electronic watermark data in portions not adversely affecting the image quality, such as LSBs of pixel data. However, this method may remove the electronic watermark data from images. For example, information regarding LSBs of pixels will be missed using a low-pass filter. The image compression process discards the volume of information not adversely affecting the image quality, thus reducing the volume of data. This means that the electronic watermark data is lost. As a result, the problem is that it is difficult to re-detect the electronic watermark data.
JP-A-No. 315131/1994 discloses as another example the method of burying specific information by using the correlation between continuous frame images and detecting the area where degradation in image quality does not occur even when substitution is performed in peripheral areas upon reproduction. FIG. 9 shows the method of inserting and detecting electronic watermark data, disclosed in the above publication. According to this method, an identification data buried area is specified signal dropout portion and conversion information upon reproduction and then the corresponding portion is corrected to reconstitute an Image.
As further another example, JP-A-No. 30466/1993 discloses the method of converting the frequency of a video signal and then burying information with signals of frequencies lower than the frequency band of the converted video signal. FIG. 10 shows the electronic watermark detecting system, disclosed in the above publication. In this method, a broad band-pass filter extracts the original video signal while a low-pass filter extracts the buried identification data.
In another example, the method of frequency-converting images and then burying electronic watermark data into portions with strong frequency components of a video signal after the frequency conversion (see “NIKKEI Electronics”, 1996, 4.22 (no. 660), page 13).
In this method, since electronic watermark data is buried into areas with strong frequency components, the electronic watermark data is not lost through the image processing such as compression process or filtering. Moreover, using the random numbers with a normal distribution as electronic watermark data makes it difficult to prevent interference between electronic watermark data and to destroy the electronic watermark data without significantly affecting the entire image.
In the electronic watermark data burying method, the original image is first transformed into frequency components by, for example, the DCT (discrete cosine transform). n sets of data with high values over high frequency range are selected as f(1), f(2), . . . , f(n). The electronic watermark data sets, w(1), w(2), . . . w(n), are selected from a normal distribution having an average of 0 and a dispersion of 1. The formula, F(i)=f(i) +α×|f(i)|×w(i), where α is a scaling factor, is calculated to obtain respective (i)s. The frequency component in which f(i) is substituted for F(i) undergoes IDCT (inverse discrete cosine transform) so that the image in which the electronic watermark data is buried is obtained.
Moreover, the electronic watermark data is detected according to the following method. In this detection method, both the original image and electronic watermark data candidate w(i) (where i=1, 2, . . . , n) must be known.
First, the image containing electronic watermark data is converted into frequency components through, for example, DCT. Values corresponding to factor values, f(1), f(2), f(n), each containing an electronic watermark, are set as f(1), F(2), . . . , F(n), respectively. The formula, W(i)=(F(i) −f(i))/f(i), is solved using f(i) and F(i) to extract the electronic watermark data W(i).
Next, the statistical similarity C between w(i) and W(i) is obtained by the following formula including a vector inner product.C=W×w/(WD×wD)where W=(W(1), W(2), . . . , W(n)); w=(w(1), w(2), . . . , w(n); WD=the absolute value of a vector W; and wD=the absolute value of a vector w. When the statistical similarity C is more than a specific value, it is decided that the electronic watermark data is in a buried state.
Accordingly, by burying the electronic watermark data into an image through the above-mentioned method, an author holding an original image effectively detects digital image data suspected as an illegal replicate. Since the above-mentioned method requires the original image, the author, that is, an original image owner, can detect image data doubted as an illegal replicate. However, the reproduction unit at each terminal cannot detect electronic watermark data because of the absence of the original image.
To overcome that problem, an improvement of the above-mentioned method for the terminal processing, particularly, the MPEG system, has been proposed. In this improved method, the original image is divided into blocks each having 8 pixels×8 pixels. Electronic watermark data is buried or extracted in block process units.
In the electronic watermark data burying process, AC frequency components are set as f(1), f(2), . . . , f(n) in a frequency increasing order over a frequency range after the discrete cosine transformation in the MPEG encoding process. The electronic watermark data, w(1), w(2), . . . , w(n) are selected from the normal distribution having an average of 0 and a dispersion of 1. In order to obtain respective (i)s, the formula of F(i)=f(i)+α×avg (f(i))×w(i) is calculated, where αis a scaling factor and avg(f(i)) is a partial average obtained by averaging the absolute values at three points adjacent to f(i). The successive steps in the MPEG encoding process is performed by substituting f(i) with F(i).
Electronic watermark data is detected according to the following method. This method does not require any original image. It is merely required that electronic watermark data candidate w(i) (where i=1, 2, . . . , n) is known.
Over the block frequency region after inverse quantization of the MPEG expanding process, the frequency components are set as F(1), F(2), . . . , F(n) in a frequency increasing order. The average of the absolute values of three points adjacent to F(i) is set as a partial average avg(F(i)). The electronic watermark data W(i) is obtained by calculating the following formula.W(i)=F(i)/avg(F(i))
Moreover, t he sum WF(i) of w(i) for one frame is calculated for each (i).
Next the statistical similarity C between w(i) and WF(i) is obtained by calculating the following formula including an vector inner product:C=WF×w/(WFD×wD)
where W=(WF(1), WF(2), . . . , WF(n)); w=(w(1), w(2), . . . , w(n)); WFD=the absolute value of a vector WF; and wD=the absolute value of a vector w. When the statistical similarity C is more than a specific value, it is decided that the corresponding electronic watermark data is in a buried state.
However, the above-mentioned prior-art techniques have the following disadvantages. In the example disclosed in JP-A-No. 315131/1994, because the electronic watermark information is not buried into all frames, frames with no buried electronic watermarks cannot be prevented from illegal copying. Moreover, since it is assumed that successive frames are static images and not changed, the area where electronic watermark data is buried cannot be specified so that electronic watermark data cannot be buried in fast moving images.
In the example disclosed in JP-A-No. 30466/1993, since electronic watermark data is buried in the frequency area lower than that after frequency conversion of an image, the high-frequency pass filter can easily remove the electronic watermark data.
In the example of burying the electronic watermark into the portion with strong frequency component after frequency conversion, the above-mentioned problems do not occur. However, since the common electronic watermark data is buried in any scene, the electronic watermark is easily viewed on the screen with small motion like a static image if the electronic watermark is emphasized to improve the detection efficiency. As a result, the image quality is deteriorated. Further problem is that the detection efficiency is reduced if the electronic watermark is weakly inserted to prevent deterioration in image quality.